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Abstract 

Background: The tendency to fornn conclusions based on linnited evidence is known as the 'junnping to 
conclusions' (JTC) bias, and has been a nnuch studied phenonnena in individuals with psychosis. Previous reviews 
have supported the hypothesis that a JTC bias is particularly linked to the fornnation and nnaintenance of delusions. 
A new systematic review is required as a number of studies have since been published, and older reviews are 
limited by not systematically assessing methodological quality or the role of study design in influencing effect size 
estimates. This review aimed to investigate if there is an association between psychosis or delusions and JTC bias. 

Methods: The current protocol outlines the background and methodology for this systematic review and meta-analysis. 
Eligible articles will be identified through searches of the electronic databases Psyclnfo, PubMed and Medline using relevant 
search terms, supplemented by hand-searches of references within eligible articles and key review articles within the field. 
Eligibility criteria were as follows: studies must recruit individuals with: i) schizophrenia spectrum conditions or ii) experiences 
of delusions. Case-control, cross-sectional, observational and prospective designs will be included but treatment trials and 
experimental studies excluded. Studies must use the beads task to assess JTC or a conceptually equivalent task. The 
outcomes will be the average number of 'draws to a decision' in the beads task (or related variant) and the proportion 
of the sample judged to demonstrate a JTC bias. Literature searches, study selection, data extraction, risk of bias 
assessment and outcome quality assessment will be undertaken by two independent reviewers. Meta-analyses 
will be undertaken for continuous (mean number of 'draws to a decision') and binary outcomes (number of 
people classified as having JTC bias). 

Discussion: Understanding of the size of the JTC effect and the contexts within which it occurs is important 
both in terms of informing models of delusional thinking and in guiding treatments for those with delusions or 
psychosis. However, a definitive, up-to-date review and meta-analysis of the JTC bias is currently lacking. The 
proposed review will fill this gap and resolve key issues regarding the factors which moderate the JTC bias. 

PROSPERO registration: CRD4201 4007603 http://www.crd.york.ac.uk/PROSPERO/display_record.asp? 
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Background 

There is evidence that people with delusional beliefs, usu- 
ally in the context of a psychotic illness, make decisions 
on the basis of less evidence than people without such be- 
liefs [1]. Consequently, people with delusions have been 
described as having a 'jumping to conclusions' (JTC) bias 
[2]. Such reasoning biases may contribute to the forma- 
tion of a delusional belief in so far as the belief is formed 
on the basis of little evidence, without considering alterna- 
tives, or looking for further information [3]. 

This JTC finding has been demonstrated repeatedly, 
usually on a measure of probabilistic reasoning called 
the beads task. In this task, participants are presented 
with two jars containing coloured beads in equal but op- 
posite ratios. For example, the jars may contain 85 black 
beads and 15 red and vice versa. Participants are told 
that one of the jars has been chosen and beads from the 
jar will be presented one at a time, and that their task is 
to decide when they know with certainty whether it is 
the mainly black bead jar or the mainly red bead jar that 
has been selected. 

A number of studies have suggested that, as a group, 
people with delusions require fewer beads before making 
a decision, or fewer 'draws to a decision' (DTD), than 
clinical and non-clinical controls [2,4]. A second out- 
come arising from the beads task is extreme responding 
(or JTC), which has been defined as deciding on the 
basis of two or fewer beads [5]. According to this criter- 
ion, around 40 to 70% of people with delusions seem to 
demonstrate JTC, compared to around 10 to 20% of 
clinical and non-clinical participants [6]. Similar biases 
have been shown on other tasks that also involve the 
gathering of information required for a decision [7,8]. 

The beads task typically uses material unrelated to the 
content of the delusional belief. This is valuable as if 
people were to reason about their delusional beliefs it 
could lead to tautological explanations. However, clearly 
delusions tend to have a limited number of themes, and 
in the case of grandiose and persecutory delusional be- 
liefs which are particularly common, the theme is 
around whether a person is liked or valued, and is essen- 
tially about their worth [9]. Consequently, some re- 
searchers have manipulated the content of the beads 
task to reflect more emotionally salient themes (for ex- 
ample, indicating that the task is to decide if a person 
very much like yourself is liked by other people'). Using 
such methods there has been some indication that it 
may increase the data gathering bias and/or lead to more 
errors in decision- making [1]. 

The suggestion that delusions may involve a hasty data 
gathering style, which precludes consideration of alternative 
explanations, has been incorporated in a number of multi- 
factorial models of delusion formation and maintenance; 
for example, see [9]. Consequently, this data gathering style 



is specifically targeted in a number of emerging treatment 
packages in order to help people overcome this style and 
possibly to improve the efficacy of Cognitive Behavioural 
Therapy (CBT) for delusional beliefs [10]. 

In the only existing meta-analysis. Fine and colleagues 
[4] concluded that the best measure of the data gather- 
ing bias is DTD. They also noted that a tendency to 
gather less evidence in the beads task is reliably associ- 
ated with the presence of delusional symptomatology, 
but that emotionally salient content had no discernible 
effect on data gathering. This was a helpful review but 
was limited in a number of ways. First, there was a lim- 
ited literature available at the time. There were 12 clinical 
studies, and two non-clinical studies (using participants 
who scored high or low for delusional ideation). Similarly, 
in relation to the impact of content on decision-making 
there were only three studies using salient materials. Sec- 
ond, at that time the studies were characterised by small 
sample sizes and, third, there was no consideration of 
methodological quality. This is problematic as guidelines 
for reporting in meta-analyses emphasise the importance 
of methodological quality assessment [11] as this can have 
a substantial impact on the interpretation of results and 
the conclusions drawn. Fourth, the meta-analysis violated 
the assumptions of statistical independence by combining 
non-independent effects, which is advised against due 
to the impact on the standard error of the aggregated 
effect [12]. 

Since that review, there has been a proliferation of 
studies with some 40 or so being reported using clinical 
samples and some 12 or more using non-clinical partici- 
pants [3]. The sample sizes in these studies have in- 
creased [13]. In addition, the range of groups that have 
been considered has expanded so that studies have been 
undertaken with people in their first psychotic episode 
[14], as well people in more acute, chronic and remitting 
states [15,16]. 

A recent review updated the evidence but did not de- 
ploy meta-analysis or investigate whether methodological 
quality moderated the findings [3]. Hence it would seem 
timely to revisit the literature. The goal of the proposed 
review and meta-analysis will be to provide a robust and 
up-to-date account of the true size of the JTC effect in re- 
lation to delusions and psychosis, considering a variety 
of theoretically informed moderators and providing an 
assessment of the effect of study quality on overall 
estimates. 

Aims and scope of this review 

The main aim of this review is to consider the extent to 
which people with non-affective psychosis, and specifically 
those with delusions, demonstrate reduced data gathering 
relative to psychiatric and non-clinical controls. We will 
investigate (a) whether there are a fewer number of DTD 
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for individuals diagnosed with a psychotic disorder versus 
individuals without these experiences, (b) whether signifi- 
cantly more people with non-affective psychosis demon- 
strate a JTC style relative to control participants; (c) a key 
question is whether the bias is specific to delusions, or 
whether it is more general to psychosis, and we will there- 
fore also test whether DTD and JTC style differ between 
those experiencing delusions versus people who have 
psychosis but do not report delusions. 

Two broad sets of secondary questions will address a) 
the impact of illness variables and b) the impact of task 
variables on data gathering in people with non- affective 
psychosis. Since the focus of this review is on people 
with clinical levels of psychosis, we will not examine dif- 
ferences in non-clinical participants who are high and 
low in delusional ideation (this will be subject to a separ- 
ate review; Taylor, Hutton and Dudley, in preparation). 

With regards the task variables, performance by people 
with and without non-affective psychosis will be consid- 
ered on easy versions of the beads task which will be de- 
fined as tasks using 85:15 ratio, and in a slightly broader 
criteria those studies using ratios of 90:10, and 80:20. Con- 
versely, 'hard' versions of the beads tasks are defined as 
those that use the 60:40 ratio [4]. Additionally, the effect 
of salient or emotional variants of the tasks will also be 
considered [1]. Methodological factors, such as whether 
there are practice trials or multiple trials, will be ad- 
dressed. Finally, whether an experimenter is present at the 
time of testing or whether the task is fully automated will 
be considered to test the hypothesis that raters might 
inadvertently cue an early response from people with 
psychosis, as in the 'Clever Hans' phenomenon; [17]. 
In summary, the moderators of effect size we intend to 
test are as follows: 

• Whether the participants in the psychosis group 
also have delusions (delusions present versus absent) 

• The stage of the psychosis (early psychosis versus 
chronic psychosis) 

• Whether the participants were reporting current 
delusions or were remitted 

• The difficulty of the JTC task (easy versus difficult task) 

• The saliency of the JTC task (emotionally salient 
versus standard JTC task) 

• The blinding of the researcher during JTC task 
(blind-rater or automated task versus rater aware of 
participant diagnosis/delusions) 

• The matching of participants on demographics 

• Whether the JTC task employed practice trials and/ 
or multiple trials. 

Method 

The review will aim to adhere to the Preferred Reporting 
Items for Systematic Reviews and Meta- Analyses (PRISMA) 



guidelines, [18] and the Assessment of Multiple Systematic 
Reviews (AMSTAR) guidelines [11] in order to ensure com- 
prehensive and transparent reporting throughout. 

Inclusion and exclusion criteria 
Population 

Studies are required to recruit a sample of individuals ei- 
ther (i) with a diagnosis {International Classification of 
Diseases, version 10 (ICD-10) or Diagnostic and Statis- 
tical Manual of Mental Disorders, version 4 (DSM-IV)) 
of a schizophrenia spectrum condition (for example, 
schizophrenia, schizoaffective disorder, schizophreni- 
form disorder, psychosis not otherwise specified (NOS)) 
or (ii) who experience delusions irrespective of condition 
(that is, trans-diagnostically). Including both types of sam- 
ple enables us to explore the JTC bias at the more general 
level of psychosis versus non-psychosis and at the more 
specific level of delusions versus no delusions. This is cru- 
cial in establishing whether the JTC bias is specific to de- 
lusions per se or a correlate of psychotic illness more 
generally. The presence or absence of delusions may be 
based upon structured clinical interviews, such as the 
Positive And Negative Symptom Scale (PANSS); [19], or 
self-report measures, which may be more limited in terms 
of validity and reliability. The way in which delusions are 
assessed will be taken into account during the quality as- 
sessment (see below). 

Studies will be excluded where > 50% of the sample 
also have co-morbid diagnoses of bipolar disorder, learn- 
ing disability, a primary diagnosis of substance-induced 
psychosis or psychosis secondary to a general medical 
condition or organic pathology. 

For the purposes of the review we define first-episode 
or early psychosis as a first diagnosis of a schizophrenia- 
spectrum condition occurring within the last two years, 
following other research in this area [20]. 

The control groups may include both healthy controls 
and individuals with psychiatric conditions other than 
psychosis. Furthermore, in the cases of analyses compar- 
ing individuals with and without experiences of delusions, 
controls may include individuals diagnosed with psychotic 
disorders but who have not experienced delusions. 

Design 

A range of study designs will be suitable for inclusion. 
These include case-control studies whereby the cases 
may be defined either by the presence or absence of 
psychosis/delusions (where JTC bias would be the out- 
come), or by the presence or absence of a JTC bias 
(where psychosis or delusions would be the outcome). 
Cross -sectional correlational studies, whereby the rela- 
tionship between JTC bias and continuous dimensions 
of delusional experience (for example, frequency, sever- 
ity) is tested will be also be considered. Prospective 
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designs where the relationship between psychosis or de- 
lusions and JTC bias is examined over time would also 
be eligible. Experimental designs whereby JTC bias is ex- 
perimentally manipulated or trials of interventions aimed 
at improving JTC bias will be excluded. We will explore 
the impact of different study designs through sensitivity 
analyses. 

Assessments of JTC bias 

Studies employing either the beads task [21] or concep- 
tual variants of this approach, for example, [22] will be 
included. Studies must report a measure of either (i) 
DTD, or (ii) a categorical index of JTC bias based on 
Garety and colleagues' definition [5]. Studies that do not 
measure JTC but instead include a measure of 'jumping 
to perceptions' [7] or 'jumping to attributions' [8] will be 
excluded, as these tasks are not conceptual equivalents 
of the beads task. 

Additional criteria 

Only English language articles will be included. Unpub- 
lished data will also be included where identified. 

Search strategy 

Electronic databases, including Psychlnfo, PubMed, Medline 
and Web of Science, will be searched using the following 
keywords: (JTC or 'jumping to conclusions' or 'jump to 
conclusions' or 'data gathering' or beads or 'probab* 
reason*') AND (psycho* or schiz* or paranoi* or delu- 
sion* or persecut*). Searches will be undertaken using 
OVID search tools. Searches will go back as far as 
1988, as this coincides with the first published study to 
use the beads task within psychosis (Huq et aL [21]). 
The electronic database searches will be undertaken 
independently by two authors (PJT, PH). Both authors 
are research psychologists specialising in clinical psychology 
with past experience of undertaking systematic reviews and 
meta-analyses. 

References will be searched for all papers included in 
the review in order to identify any further relevant stud- 
ies. In addition, the reviews of the JTC bias literature by 
Fine, and colleagues (2007), Dudley and Over [2], Garety 
and Freeman [3,23] and Freeman [9], will be examined 
to determine if any additional studies are included that 
were missed by our own literature searches. As a final 
step, all corresponding authors of included articles will 
be contacted and asked if they are aware of any further 
studies potentially meeting our criteria, including both 
recently published and unpublished studies. 

Study selection 

In the initial phase, titles and abstracts will be screened 
for potentially eligible studies. In the second phase, full 
texts of the remaining articles will be read to determine 



if they meet the inclusion and exclusion criteria (all 
screening by PJT and PH). Where disagreement emerges 
regarding the eligibility of studies, a third author (RD) 
will arbitrate. Where conference abstracts that are iden- 
tified through the search that possibly meet the inclusion 
criteria, the presenters will be contacted and further de- 
tails sought regarding the study in order to ascertain the 
studies eligibility. 

Data extraction 

Data will be independently extracted by two authors 
(PJT and PH) using a standard data collection form. Re- 
sults will be compared and consistencies resolved with a 
third author (RD) acting as an arbitrator. Collected data 
will include sample characteristics (gender, age, ethnicity, 
cUnical diagnosis, stage of illness, sample source and 
location), study design, JTC task (including bead ratio, 
number of trials, researcher blind or not, automated 
procedure or not), outcome data (for example, means, 
standard deviations, proportions of participants exhi- 
biting JTC bias, correlations and regression weights 
where applicable). 

Methodological quality 

We have adapted a tool for assessing the methodological 
quality of observational studies that has been successfully 
employed in prior research undertaken by the Agency for 
Healthcare Research and Quality (AHRQ) [24]. The main 
methodological quality criteria were retained but the 
underlying factors related to each study quality criterion 
were adapted in some instances for this specific context. 
Each study is assessed on a number of methodological 
quality criteria (for example, unbiased selection of groups, 
sample-size calculations, and so on) that are rated as being 
met, not met, partially met, or being unclear. A copy of 
this adapted measure is presented in Appendix 1. 

Following the guidance of experts in the field of meta- 
analysis, we will avoid scale-based or aggregated study 
quality rating [25]. Quality assessments will be presented 
descriptively to guide the interpretation of findings, ra- 
ther than used as a means to weight or adjust aggregated 
effect sizes. However, as noted, specific aspects of meth- 
odology will be tested as moderators of effect sizes. 
These will include blinding and the matching of partici- 
pants on demographics. 

Assessing overall quality of evidence 

The GRADE approach [26] was also adopted to provide 
an assessment of quality at the outcome level. Within 
the GRADE approach, observational studies are nor- 
mally automatically marked down for quality. However, 
for the purposes of this review, as all included studies 
will be observational, a decision was made not to auto- 
matically mark down the quality of outcomes in this 
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manner. All outcomes are therefore initially rated as 
high quality and then downgraded based on five main 
criteria (risk of bias, imprecision, indirectness, hetero- 
geneity, publication bias). 

The methodological quality of all studies and outcomes 
will be assessed independently by two authors (PJT and 
PH), with RD again acting as arbitrator. Both PJT and PH 
have completed the GRADE online learning modules 
(http://cebgrade.mcmaster.ca) and have experience of 
using this approach. 

Data synthesis and analysis 

For studies comparing DTD between groups. Hedges s g 
will be calculated, alongside the raw mean difference, as 
this index corrects for small sample bias [27]. Where 
means or standard deviations are not available to calcu- 
late effect size and associated standard error this will be 
estimated from other reported statistics (for example, f) 
or authors will be contacted for this information. Where 
JTC bias is treated as a binary variable, odds ratios (OR) 
will be calculated as an index of effect size. 

Other research designs require additional consider- 
ation. Mean symptom severity data from studies com- 
paring JTC and non-JTC clinical groups will not be 
compatible with studies comparing groups with and with- 
out psychosis/delusions. Where present, these data will be 
converted into OR, which will be used in the meta-analysis 
of binary outcomes. For studies employing correlational 
analyses, these will be converted to Hedgess g (formula 
from Borenstein et al [12]). Sensitivity analyses will be 
used to explore the impact of combining these converted 
effect sizes with other studies for the primary outcomes. 

Where studies provide multiple comparisons between 
three or more different groups (for example, a group with 
psychosis, a psychiatric control group and non-clinical 
control group) a combined effect size will be estimated 
taking into account the correlation between effects (for- 
mula from Borenstein et al [12]), and used in the 
meta-analyses. In addition, we will undertake a sensi- 
tivity analysis, re-calculating the meta-analyses to look 
specifically at comparisons with either psychiatric con- 
trols or non-clinical controls. 

The presence of statistical heterogeneity will be assessed 
via the Q-test of statistical heterogeneity and quantified 
via the [28]. A random-effects model will be calculated 
within the meta-analyses as some degree of heterogeneity 
is expected across the studies. Nonetheless, where hetero- 
geneity is moderate or less < 40% [29], a sensitivity ana- 
lysis will be undertaken examining the difference between 
fixed-effects and random-effects models. 

Subgroup analyses 

As noted in the review aims, a number of moderators of 
effect size will be explored as part of the secondary 



outcomes of the review. These moderator effects will be 
examined through subgroup analyses. Significant differ- 
ences between subgroups will be ascertained using the 
Q-test (Borenstein et al [12]). 

Publication bias 

Publication bias will explored via funnel plots for all out- 
comes with ten or more studies, following recommenda- 
tions by Sterne, Egger and Moher [30]. The Trim and 
Fill method [29] will also be employed to explore the 
presence and ascertain the potential impact of publica- 
tion bias. All analyses will be undertaken using Compre- 
hensive Meta- Analysis [31] and STATA version 9 (Stata 
Corporation, College Park, TX, USA). 

Discussion 

People with delusions are thought to have a data gather- 
ing bias in which they make decisions based on limited 
evidence. This 'jumping to conclusions' reasoning bias is 
thought to contribute to the development and mainten- 
ance of delusions. However, the robustness of this find- 
ing needs to be subject to systematic review and the 
extent of the effect considered with meta-analytic proce- 
dures. This proposed review will act as a definitive inves- 
tigation of this well investigated process. In addition to 
quantifying the size of the JTC effect and its specificity 
to delusions versus psychosis in general, we also aim to 
determine whether a range of illness and task-related 
variables moderate the strength of the JTC effect. In 
light of this review, further development of theory and 
or clinical practice may well be warranted. In discussing 
the findings of the review, we will consider how they 
compare with the results of previous reviews in this area, 
their implications for future research and policy, the lim- 
itations and strengths of the review, and future research 
recommendations that can be drawn in light of the limi- 
tations of the available evidence. 

Appendix 1: quality assessment tool 

General instructions: Grade each criterion as 'Yes', 'No', 
'Partially', or 'Can't tell'. Factors to consider when making 
an assessment are listed under each criterion. Where ap- 
propriate (particularly when assigning a 'No', 'Partially', or 
'Can't tell' score), please provide a brief rationale for 
your decision (in parentheses) in the evidence table. 

1. Unbiased selection of the cohort? 
Factors that help reduce selection bias: 

o Inclusion/exclusion criteria: 

o Recruitment strategy 

■ Clearly described 

■ Criteria for inclusion in psychosis/delusions 
and comparison groups clearly outlined. 

o Recruitment strategy: 
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■ Clearly described 

■ Relatively free from bias (selection bias might 
be introduced, for example, by recruitment via 
advertisement). 

2. Selection minimizes baseline differences in 
prognostic factors? 

Factors to consider: 

o Was selection of the comparison group 
appropriate? 

o Is the comparison group matched with the 
clinical group on key demographics (that is age 
and gender)? 

3. Sample size calculated? 
Factors to consider: 

o Did the authors report conducting a power 
analysis or describe some other basis for determining 
the adequacy of study group sizes for the primary 
outcome(s) of interest to us? 

o Where a power calculation is presented, do the 
final numbers obtained match up to this (for 
example, within 10% of required numbers)? 

4. Adequate description of the cohort? 

Consider whether the cohort is well-characterized in 
terms of baseline: 
o Age 
o Sex 

o Ethnicity 

o Diagnosis/clinical status 

5. Validated method for ascertaining psychotic 
disorder or delusions? 

Factors to consider: 

o Was the method used to ascertain exposure 
clearly described (details should be sufficient to 
permit replication in new studies)? 
o Was a valid and reliable measure used to 
ascertain exposure (subjective measures based on 
self-report tend to have lower reliability and validity 
than objective measures such as clinical interview)? 
Likewise, relying on medical notes is likely to 
introduce bias due to variation in how assess- 
ment is undertaken. 

6. Validated method for ascertaining 'jumping to 
conclusions'? 

Factors to consider: 

o The beads task or a conceptually equivalent 
variant should be used 

o Were these measures implemented consistently 
across all study participants? 
o Were several trials and/or a practice run 
included in the procedure? 

7. Outcome assessment blind to exposure? 
Factors to consider: 

o Were the study investigators who assessed 
outcomes blind to whether participants had a 



psychotic disorder or delusions (this criterion will 
not apply in the case of Internet-based or auto- 
mated designs where a researcher is not present)? 

8. Adequate handling of missing data? 
Factors to consider: 

o Are the details of missing data clearly reported, 
including how missing data was handled in the 
analyses? If not, is there any reason to believe 
missing data was present (for example, lower N in 
analysis than initially reported in the participants 
section). 

o Did missing data from any group exceed 20%? 
o If missing data was present and substantial, 
were steps taken to minimize bias (for example, 
sensitivity analysis or imputation). 

9. Analysis controls for confounding? 
Factors to consider for controlled studies: 

o If groups were not matched as baseline, did the 
analysis control for any baseline differences between 
groups? 

o Does the study identify and control for important 
confounding variables and effect modifiers (for 
example, IQ)? 

10. Analytic methods appropriate? 
Factors to consider: 

o Was the kind of analysis done appropriate for 
the kind of outcome data (categorical, continuous, 
and so on)? 

o Was the number of variables used in the analysis 
appropriate for the sample size (the statistical 
techniques used must be appropriate to the data 
and take into account issues such as controlling 
for small sample size, clustering, rare outcomes, 
multiple comparison, and number of covariates 
for a given sample size)? 

Abbreviations 

JTC: Jumping to conclusions; DTD: Draws to decision; CBT: Cognitive 
Behavioural Therapy. 
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